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What is claimed is: 

l\ A method of classifying data according to perceptual properties of the data, the 
memod being suited for searching and sorting large databases of media entities, including 
musiV video and image databases, the method comprising: 

\assigning to each media entity of a plurality of media entities in a data set to at least 
one classy each class of said at least one class corresponding to a subset of perceptual 
properties We-defined for the data set; 

processing each media entity of said data set to extract at least one digital signal 
processing characteristic for each media entity; 

generatiiTG a plurality of feature vectors for said plurality of media entities, wherein 
each vector incluates said at least one class and said at least one digital signal processing 
characteristic; and \ 

forming a classification chain based upon said plurality of feature vectors. 

2. A method according to claim 1 , further comprising calculating a neighborhood 
distance within the vector «pace of said classification chain for each of said at least one 
perceptual class. \ 

3. A method according toVlaim 2, wherein said neighborhood distance defines a 
distance within which two vectoiB in the classification chain space are in the same 
neighborhood. \ 

4. A method according to claim 4, wherein said calculating of a neighborhood distance 
for each of said at least one perceptual Class includes determining a distance within which two 
vectors of the classification chain possessvthe same class given a threshold degree of error. 

5. A method according to claim 4, wherein a human determines said threshold degree of 
error that will be tolerated, said threshold degree of error placing a maximum limit on the 
distance that may be used for determining neighbprhoods. 

6. A method according to claim 1, further comprising: 
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. processing an unclassified media entity to extract at least one digital signal processing 
chaWteristic for the unclassified media entity; 

\ generating a vector for the unclassified media entity including said at least one digital 
signal processing characteristic; 

presenting the vector for the unclassified media entity to the classification chain; and 
classifying the unclassified entry with an estimate of at least one perceptual class by 
calculating trb representative at least one perceptual class of the subset of the plurality of 
vectors of the classification chain located in the neighborhood of the vector for the 
unclassified entity. 

7. A method according to claim 6, wherein said classifying of the unclassified entry 
includes locating the subset of the plurality of vectors of the classification chain located in the 
neighborhood. \ 

8. A method according to claim 7, wherein said locating includes calculating, as the set 
of vectors of the plurality of vectors of the classification chain within the neighborhood 
distance, the set of vectors of th^ plurality of vectors of the classification chain within a 
maximum Euclidian distance for massifying new unclassified entities within a threshold 
tolerance. \ 

9. A method according to claim sXwherein a human determines said maximum 
Euclidian distance for purposes of determining the neighborhood vectors within the threshold 
tolerance. \ 

10. A method according to claim 6, whereui said classifying of the unclassified entry 
includes classifying the unclassified entry with a median class represented by the 
neighborhood. \ 

11. A method according to claim 6, wherein saimclass is described by a numerical value 
and said classifying of the unclassified entry includes Classifying the unclassified entry with a 
mean of numerical values found in the neighborhood. \ 
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i^. A method according to claim 6, wherein said classifying includes returning at least 
one number indicating the level of confidence of the class assignment. 

13. \ A method according to claim 12, wherein said classifying includes returning a ratio 
definedVy the number of entries in the neighborhood having the representative class over the 
number oKentries in the neighborhood not having the representative class. 

14. A metnod according to claim 12, wherein said class is described by a numerical value 
and said classifying includes returning a standard deviation of the numerical values in the 
neighborhood. \ 

15. A computer reaMble medium bearing computer executable instructions for carrying 
out the method of claim K 

16. A modulated data signal carrying computer executable instructions for performing the 
method of claim 1 . \ 

17. A computing device comprising means for performing the method of claim 1 . 

18. A computing system, comprising^ 

a computing device including: \ 

a classification chain data structure stored thereon having a plurality of classification 
vectors, wherein each vector includes data representative of at least one perceptual class as 
classified by humans and digital signal processing data as classified by at least one computing 
device; and \ 

processing means for comparing an unclassified media entity to the classification 
chain data structure to determine at least one perceptual class of said unclassified media 
entity. \ 

19. A computing system according to claim 18, wherein said comparing includes 
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Vietermining a set of classification vectors that are within a neighborhood distance fi'om a 
vector represented by said unclassified media entity. 

20. \a computing system according to claim 19, wherein said neighborhood distance 
defines aMistance within which two vectors in the classification chain space are in the sane 
neighborhocid. 

21. A computing system according to claim 20, wherein said neighborhood distance is 
determined for eacli of said at least one perceptual class, and said determination includes 
determining a distance within which two vectors of the classification chain data structure 
possess the same class given a threshold degree of error. 

22. A computing device according to claim 2 1 , wherein a human determines said 
threshold degree of error that will be tolerated, said threshold degree of error placing a 
maximum limit on the distance tnat may be used for determining neighborhoods. 

23. A computing system according, to claim 19, wherein said processing means classifies 
the unclassified entry with an estimate oEat least one perceptual class based upon the vectors 
within the neighborhood of a vector representing the unclassified entity. 

24. A computing system according to claimV3, wherein said classifying includes 
classifying the unclassified entry with a median cmss represented by the neighborhood. 

25. A computing system according to claim 23, wherein said at least one perceptual class 
is described by a numerical value and said classifying oftiie unclassified entry includes 
classifying the unclassified entry with a mean of numericalValues found in the neighborhood. 

26. A computing system according to claim 23, wherein saidclassifying includes 
returning at least one number indicating the level of confidence oiSthe class assignment. 

27. A computing system according to claim 26, wherein said classifying indudes 
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^turning a ratio defined by the number of entries in the neighborhood having the 
reptesentative class over the number of entries in the neighborhood not having the 
representative class. 

28. A computing system according to claim 26, wherein said class is described by a 
numerical vaUie and said classifying includes returning a standard deviation of the numerical 
values in the neighborhood. 

29. A computing, system according to claim 18, further comprising: 
an input deviceJTor receiving a new unclassified media entity; and 

an output device tor outputting at least one perceptual class of said new unclassified 
media entity based upon said classification chain based upon processing of said processing 
means. \ 

30. A classification chain data\tructure utilized in connection with the classification of 
new unclassified media entities, conitorising: 

a plurality of classification vectbrs, wherein each vector includes: 
perceptual data classified humans; and 

digital signal processing dataVclassified by at least one computing device. 

31. A method of generating a classification cnain having a plurality of vectors describing 
a plurality of media entities, comprising: \ 

assigning, by an expert, a first value to a medm entity according to a pre-defined 
perceptual characteristic of media entities; \ 

assigning, by a computing system, a second value\p the media entity according to a 
pre-defined digital signal processing characteristic; \ 

generating a vector based on at least said first value arwi said second value; and 

adding said vector to a classification chain data structureX 

32. A method according to claim 3 1 , wherein said method is repeated until a sufficient 
number of vectors have been added to said classification chain data structure, such that said 
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classification chain data structure successfully classifies unclassified media entities within a 
thresliold degree of success. 

33. ASmethod according to claim 32, further comprising: 

inputting a vector representative of a new unclassified media entity; 
comparing said vector to the vector space of the classification chain data structure; 
and \ 

outputting\n estimate of the perceptual class of the new unclassified media entity 
based upon vectors round in a neighborhood of the classification chain data structure, wherein 
a neighborhood of vectors is defined as a set of vectors located within a neighborhood 
distance. \ 

34. A method according tCi claim 33, further comprising: 

outputting an estimate o^die confidence level with which said estimate of the 
perceptual class is correct. \ 

35. A method according to claim 34, wherein if said estimate of the confidence level is 
low, a human examines the new unclassified media entity and said human determines an 
action for said classification chain data structure based thereon. 

36. A method according to claim 35, wherein said human adds a new subset of perceptual 
properties to the classification chain data structure defined by the new unclassified data 
structure. \ 

37. A method according to claim 35, wherein said miman modifies an existing subset of 
perceptual properties represented by the classification chVin data structure in accordance with 
the results of the human examination. \ 

38. A method according to claim 35, wherein the new unclassified media entity is rejected 
from the classification chain as an outlier. ^ 
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39. lA. method according to claim 35, wherein the performance level of the classification 
chain improves over time due to the examination of unclassified media entities that have a 
low confiotence level associated therewith. 



40. A metnod according to claim 33, wherein it is determined whether said vector 
representing saidnew unclassified media entity has been previously processed, thereby 
obviating the need\o re-process said vector via the classification chain. 



